quote:
DETRENDING ALGORITHM INTRODUCTION:LXSoftware Version 9.9.7 contains a custom detrending algorithm that attempts to remove long-term trends without affecting short-term reactions. The algorithm design attempts to provide the recentering capability of the existing automatic filter on the recovery side of a reaction while providing minimal attenuation during the rise side. The detrending algorithm has been designed to solve several EDA signal quality issues reported by field users. These issues include:
1)Significant baseline wander can cause the EDA trace to leave the viewable area of the screen. This makes scoring the trace very difficult. Using automatic mode (high-pass filter) fixes this problem; however, applying the legacy high-pass filter algorithm to the data can result in the attenuation of some types of reactions (particularly complex reactions and reactions of similar magnitude but differing durations). This attenuation can cause the reaction’s appearance to change in manual versus automatic mode.
2)In certain cases, the EDA trace can exhibit a significant upward or downward trend that can continue for the duration of the chart (or longer). This type of trend can cause the trace to leave the viewable area of the screen (top or bottom) within several minutes, making the trace extremely difficult to score without continual manual recentering.
DETRENDING ALGORITHM DESIGN:
The basis for the detrending algorithm is a moving average "window" whose length is configurable up to 250 points (approximarely two seconds of data). As each raw data point is processed, the history buffer is shifted and the new point is added to the end of the buffer. When the buffer is at least half full, the average (instantaneous) slope of the points within the buffer is calculated and the long-term trace average is updated as well. Increasing the number of points used in the window reduces the amount of instantaneous correction applied by the algorithm because short-term excursions tend to be averaged out of the calculations. In addition, since the “window” points prior to the start of the chart must be stored in order to correctly reproduce the appearance of the chart in review mode, a significant amount of experimentation was performed to determine the correct number of “window” points. Currently, the algorithm uses 50 data points to calculate the “window”. This value appears to provide the best filter performance while attempting to preserve the appearance of the features.
In order to minimally affect the fidelity of the reactions, the detrending algorithm implements a hystersis box within which no attenuation is performed. On well-behaved traces (e.g., those staying entirely within the hysteresis box), the algorithm does nothing. When the trace leaves the hysteresis box (top or bottom), a correction factor is applied to the waveform according to a two-stage process. The correction value is a double precision number, since the correction factor is updated incrementally. The initial correction value is 0.0. The conjugate of the instantaneous slope is calculated and the conjugate value is multiplied by a gain factor to determine the instantaneous correction value. For a trace that leaves the hystersis box and has an instantaneous slope of 0.5, the conjugate value will be -0.5. The gain multiplier represents a percentage of the long-term slope that limits the extent of the correction that is applied. The initial multiplier is typically set to 1.0. Thus, in the above scenario, if the instantaneous conjugate slope is -0.5 but the long-term slope is 0.25, the maximum correction factor will be limited to the absolute value of the long-term slope multiplied by the gain multiplier.
max_correction = +/- abs( [long-term slope] * [multiplier] ))
or, in the above case, the maximum instantaneous correction factor will be -0.25. This value is added to the double precision long-term correction factor that is subsequently added to each raw data point. The long-term correction factor serves as a baseline bias that can be positive or negative. Note the negative (lower) hysteresis value is currently set to zero to restrict the trace from going significantly below the baseline except in extreme conditions.
Adding the conjugate of the instantaneous slope stabilizes the waveform; however, it does not necessarily bias the trace back toward the baseline. In order to accomplish this task, the gain multiplier is incremented very slightly each time the raw data point is outside the hysteresis box, up to a maximum multiplier. The multiplier is decremented slightly when the raw data value falls within the hysteresis box, subject to a minimum value. This mechanism serves as a secondary throttle to gently bias the trace back toward the hystersis box.
In order to prevent the algorithm from performing open-loop corrections (e.g., attempting to correct loss of signal, etc.), there are several safety valves built into the design. First, the moving average window can be instructed to omit any points that exhibit a delta that exceeds a specified value. This can be used to exclude large reactions and/or loss of signal situations. Next, the configurable maximum correction slope value prevents overcorrection of "normal" waveforms. To fix the "plunge" situation, there are additional thresholds that bypass the maximum correction value for traces that exhibit abnormally high (but not vertical) slopes.
Finally, if the instantaneous slope of the line is zero but the trace is not centered, a small constant value is added to the correction value to attempt to bring the trace back to the baseline. This is useful in restoring the baseline when the signal is lost, etc. The constant value is very small to prevent it from adversely biasing the normal operation of the algorithm.
DETRENDING ALGORITHM STRENGTHS:
The detrending algorithm successfully corrects most commonly-reported EDA issues, including the significant EDA wander experienced by some users. While any algorithm applied to the raw data can result in appearance changes, the LXSoftware 9.9.7 detrending algorithm attempts to preserve the rising edge of the response while applying correction during the recovery phase. In the following example (provided by a LXSoftware user), the green trace is the raw EDA and the blue trace is the detrended EDA. Notice the recovery side of the waveform is heavily corrected, while the response side is almost completely unattenuated. This type of correction results in a centered trace without the universal attenuation associated with the legacy automatic (high-pass) filter.
DETRENDING ALGORITHM ARTIFACTS:
As previously mentioned, the detrending algorithm attempts to correct only during the recovery side of a reaction. Unfortunately, determining when to apply the correction can be difficult. Since the algorithm is intended to operate in real-time, it uses a relatively small snapshot of the data stream to determine if the waveform is in recovery mode. When recovery mode is detected, the algorithm increases the correction factor to bias the trace back to the baseline. Due to the relatively small window used by the algorithm, the recovery side complex reactions will be biased by the algorithm, causing the second and subsequent reactions to be biased toward the baseline. This effect is visible in the last reaction in the preceding sample chart. This effect is almost always encountered with filters that attempt to bias data to a baseline value. The amount of recovery-side attenuation can be controlled by modifying the window size and secondary throttle settings; unfortunately, lengthening the window or reducing the throttle significantly compromises the algorithm’s ability to re-center the trace. By displaying the raw and detrended data on the screen simultaneously, it is easy for an examiner to compare the detrended trace to the raw trace and determine whether or not the detrended data attenuates a reaction in an undesirable manner.